This invention relates generally to the area of nonmanual human control of external systems. More specifically, the invention provides a control system with which a user may control an external device with control signals produced in response to a collective manifestation of electroencephalographic (xe2x80x9cEEGxe2x80x9d), electromyograplhic (xe2x80x9cEM Gxe2x80x9d), electrooculargraphic and other biopotentials.
Examples of apparatus and methods for controlling devices exclusively with brain and body biopotentials are known and described In U.S. Pat. Nos. 5,474,092 and 5,692,517. In those embodiments, a user wears a headband containing three sensors for detecting brain and body signals at the forehead. The signals are transmitted to an xe2x80x9cinterface unitxe2x80x9d w here they are amplified, digitized and decoded to produce multiple command signals that users can learn to modulate consciously to control machines. It is possible with this system to achieve control of multiple functions.
However, the use of this approach has identified the need for additional capabilities. To make this technology more accessible and useful for a greater number and diversity of individuals, there is a need for the graphical user interface (GUI) and the requisite brain and body control skills to become more intuitive, comprehensible and easily mastered. There is a further need for new control signal channels for users with special needs and functional limitations. Further, novel methods are needed to identify, analyze, and apply new easy-to-produce signal characteristics and signal patterns for cursor control. For example, for an implementation of a hands-free mouse, methods to achieve left and right mouse button clicking, cursor speed and resolution switching, and activation and deactivation of the hands free mouse/keyboard controller are needed. In addition, methods by which the user can achieve user-adjustment of interface sensitivity, mouse-click and switch activation threshold, and cursor offset/bias; activation of alternative cursor horizontal motion control for wide field-of-view displays; and the triggering of keyboard commands such as xe2x80x9ctabxe2x80x9d and xe2x80x9centerxe2x80x9d are also needed.
Finally, a series of training programs, including a new music program making use of complex auditory and visual biofeedback are needed to help users learn multiple discrete and continuous control signal manipulation.
The present invention provides a comprehensive, new, flexible and sophisticated control system that is heretofore unknown. The brain-body actuated control system of the present invention has the advantage of improving the efficiency and extending the capabilities of persons engaged in hands intensive activities. The brain-body actuated control system of the present invention has the further advantage of assisting and providing additional capabilities to persons having one or more mental or physical disabilities. The brain-body actuated control system of the present invention has the advantage of being relatively inexpensive and useable to provide both continuous and discrete control signals in response to detecting user gestures.
In accordance with the principles of the present invention and the described embodiments, the invention provides an apparatus for controlling a plurality of devices in response to biopotentials produced by gestures made by a user. The apparatus includes a sensor applied to the user for producing an input signal representing the biopotentials, wherein the input signal changes as a function of changes in the biopotentials. A plurality of bandpass filters are responsive to the input signal, and each of the filters produces respective frequency signals. A processor is responsive to the frequency signals from each of the filters and generates control signals in response to gestures made by the user to command operation of the plurality of devices.
In another embodiment, the invention includes a method of controlling a device in response to biopotentials produced by gestures made by a user. The method requires detecting biopotentials at a location on a user and filtering the biopotentials with a high frequency bandpass filter to produce high frequency signals in a bandwidth of from approximately 70 Hz to approximately 3000 Hz. The method analyzes the high frequency signals with respect to a plurality of gesture models to detect a plurality of gestures being made by the user and then, provides control signals to the device for controlling the device in response to a plurality of gestures made by the user.
In one aspect of the invention, the method provides discrete control signals in response to detecting a short eyebrow lift, a long eyebrow lift, a jaw clench, a short eyebrow drop, right and left eye glances and multiple eye blink gestures of the user. In another aspect of the invention, the method provides continuous analog control signals in response to eye gestures of the user.
In a further embodiment of the invention, the brain-body actuated system is linked with a music synthesizer to produce music in response to gestures made by the user.
The brain-body actuated system of the present invention is especially useful in those applications where the user""s hands are already busy. For example, for those who are performing intensive text or data entry, the brain-body actuated controller of the present invention can be used to execute the functions of a mouse, thereby permitting the user to keep their hands on the keyboard. In other applications, a user""s hands may be performing other tasks, for example, an assembly, manufacturing, cooking cleaning or any other task, and the brain-body actuated controller can be used to perform other functions, run appliances, etc.
Similarly, in mouse intensive applications, for example, CAD applications, the brain-body actuated controller of the present invention can be used to execute various function keys, thereby not distracting the user from the mouse activity. For those who do not wish to, or are incapable of, using their hands, the brain-body actuated controller can be used in conjunction with voice recognition systems for simplifying the voice recognition application. In addition, the brain-body actuated controller has numerous applications to assist those that have one or more mental or physical disabilities in that it allows those persons to perform functions that would otherwise be beyond their capabilities.